Metrics for Threat Detection Engineering

Table of Contents

1. Introduction
2. Scope
3. Definitions
4. Metric Tiers
5. KPIs vs KRIs vs Operational Metrics
6. The Purpose and Pitfalls of Metrics
7. How to Read This Catalog
8. Primary Metrics
   • True Positive Rate (TPR)
   • False Positive Rate (FPR)
   • Mean Time to Detect (MTTD)
   • Mean Time to Respond (MTTR)
   • Automated Response Rate (ARR)
   • False Positive Reduction
9. Secondary Metrics
   • Detection Coverage
   • Detection Fidelity
   • Detection Latency
   • Incident Conversion
   • Time to Tune
   • Control Mapping Rate
   • Log Source Coverage
   • Playbook Success Rate
   • Average Playbook Execution Time
   • Integration Health Score
   • Manual Touchpoints
10. Tertiary Metrics
    • Volume Metrics
    • Detection Conversion
    • Stale Count
    • Time to Triage
    • Development Time
    • Suppression Ratio
    • Detection Effectiveness
    • Coverage Drift
    • Deployment Frequency
    • Change Frequency
    • Task Automation Ratio
    • Platform Uptime
    • Investigation Duration

Introduction

Program metrics for threat detection are quantifiable measures that evaluate how well an organization detects threats, reduces noise, and adapts to an evolving adversary landscape. They provide the visibility needed to assess detection coverage, validate engineering investments, and drive continuous improvement across a security program.

Scope

The scope encompasses all components of the threat detection framework — alerting, automation, operational processes, and response playbooks. This end-to-end coverage ensures threats are not only identified through detection rules and alerting mechanisms, but are also triaged, enriched, and escalated through automated workflows.

Definitions

The following terms are used throughout this document. Because tooling and platform vendors often use overlapping or inconsistent terminology, these definitions establish a common language for the purposes of this guide.

Detection is the process of identifying potentially malicious, unauthorized, or abnormal activity within an environment by analyzing data from sources such as logs, network traffic, endpoints, and user behavior. Detections are implemented as rules, signatures, behavioral analytics, or machine learning models that surface threats which may compromise confidentiality, integrity, or availability. Detection is the first step in the threat response lifecycle, enabling timely investigation and mitigation of security risks.

Alerts are security events generated by detections when predefined conditions are met — such as unusual login activity, malware signatures, or suspicious network behavior. An alert signals that something may require attention but does not by itself confirm malicious activity. Alerts are the raw input to the investigation process.

Tickets are formal records created to track, investigate, and manage security alerts through their lifecycle. A ticket typically includes the triggering alert, related systems or users, timestamps, analyst notes, and current investigation status. Tickets ensure accountability, enable cross-team collaboration, and provide an auditable trail from detection through resolution or escalation. They are commonly managed through case management systems, SOAR platforms, or integrated tools such as Jira or ServiceNow.

Cases are the primary unit of investigation in a modern detection program. A case correlates one or more disparate alerts into a unified, reviewable record — allowing analysts to understand related activity across different systems, users, or timeframes as a single narrative rather than isolated signals.

A note on terminology: Many platforms and teams use incident and case interchangeably, and some products use incident as the default term for what this document calls a case. The distinction matters because it reflects a shift in how detection programs think about investigation: moving from managing individual alerts to reasoning about correlated activity. Where your tooling uses incident, treat it as equivalent to case in the context of this guide. The term incident is preserved here only where it refers specifically to a confirmed threat requiring formal response — a subset of cases, not a synonym.

Playbooks are structured automation workflows attached to detections or cases that define the sequence of actions to be taken in response to a specific alert type or threat scenario. A playbook encodes institutional knowledge — triage steps, enrichment queries, escalation logic, and remediation actions — into a repeatable, auditable process. Playbooks reduce analyst decision fatigue, enforce consistency, and serve as the connective tissue between detection and response.

Automation refers to the execution of discrete actions triggered by a detection, alert, or playbook without requiring manual analyst intervention. Automation actions may include enriching an alert with threat intelligence, isolating an endpoint, blocking an IP, creating a ticket, or notifying a stakeholder. While playbooks define what should happen, automation is what actually executes. The effectiveness of automation is a measurable property of the program and a key input to several metrics in this guide.

Metric Tiers

Metrics are organized into three tiers that define audience, cadence, and purpose.

Primary

Primary metrics prove program value to leadership. They answer "is the program working?" and map directly to organizational risk reduction. Reported monthly to executives and stakeholders, these metrics should be few in number and highly trusted.

Secondary

Secondary metrics guide engineering decisions. They explain why primary metrics look the way they do and help diagnose gaps in detection coverage, fidelity, and playbook health. Reported weekly to engineering leads.

Tertiary

Tertiary metrics track operational hygiene and program maturity. Internal to the Detection Engineering team, they surface things like rule volume, stale detections, and deployment frequency — used continuously to keep the program healthy rather than to prove value externally.

KPIs vs KRIs vs Operational Metrics

Before diving into the metrics themselves, it is important to understand that not every metric serves the same purpose. Each metric in this guide is classified as one of three types — KPI, KRI, or Operational — reflecting the nature of what it measures, who it serves, and what it drives.

Key Performance Indicator (KPI)

A KPI measures how effectively you are executing. It is backward-looking or current-state — "here is what we achieved." When you report Mean Time to Detect, True Positive Rate, or playbook completion, you are answering the question "how well are we performing?"

KPIs are subdivided into three tiers that define who consumes the metric, what decision it supports, and how frequently it is reported:

• Primary — metrics that demonstrate program value to leadership, mapping directly to organizational risk reduction
• Secondary — metrics that guide engineering improvements, explaining why primary metrics look the way they do and diagnosing gaps in coverage, fidelity, and playbook health
• Tertiary — metrics that monitor platform health, operational hygiene, and program maturity over time

Key Risk Indicator (KRI)

A KRI is a forward-looking early warning signal. It measures exposure and likelihood of future failure — "here is where we might break." KRIs sit outside the tier hierarchy entirely because they serve a different function: they trigger action before a problem surfaces in your KPIs. A KRI is not reported on a cadence — it is escalated when it moves.

If a KRI starts trending in the wrong direction — a coverage gap growing, log ingestion failures increasing, detection rule staleness climbing — you want that escalated regardless of whether your MTTD and True Positive Rate still look healthy. KRIs are your canary; KPIs confirm whether the canary was right.

Operational

An Operational metric supports the day-to-day execution of the detection engineering team. It is not reported upward and does not measure risk exposure — it exists to help the team operate effectively, maintain quality, and stay self-aware. Operational metrics are internal by design and do not carry the audience or cadence expectations of a KPI.

Operational metrics matter because they represent the earliest stage of a metric's lifecycle. A metric that starts as internal team hygiene may, as the program matures, become relevant enough to report upward — at which point it gets promoted to a KPI. This promotion path is intentional. What is Operational today may be a Primary KPI in twelve months as the program grows and leadership begins asking the questions that metric answers.

Why the distinction matters in practice.

A program that tracks only KPIs is flying with instrumentation that tells you how the last flight went. KRIs tell you the engine is running hot before something fails. Operational metrics tell you whether the crew is ready to fly at all. A mature detection engineering program treats all three as essential and knows when each one should drive a conversation.

The Purpose and Pitfalls of Metrics

Metrics are a storytelling tool. They exist to communicate the health, effectiveness, and risk posture of your detection program to the people who need to act on that information. But like any tool, they can be misused — and in security programs, misused metrics don't just produce bad reports, they produce bad behavior.

Before building out a metrics program, every detection engineering team should understand two well-documented failure modes.

Goodhart's Law

"When a measure becomes a target, it ceases to be a good measure."

Goodhart's Law is the most common way metrics programs quietly degrade. It happens when the number itself becomes the goal rather than the outcome the number was meant to represent. In detection engineering this surfaces in subtle but damaging ways: a team pressured to improve Mean Time to Respond starts closing alerts faster without completing investigations. A program measured on alert volume ships low-fidelity detections to hit a number. On paper, the metrics improve. In practice, the program is moving backwards.

The fix is not to stop measuring — it is to measure outcomes, not outputs, and to pair every metric with enough context that gaming it produces no advantage. A True Positive Rate reported alongside investigation quality tells a fuller story than either number alone.

The Cobra Effect

The Cobra Effect describes what happens when an incentive produces the exact opposite of its intended result. The name comes from colonial India, where a bounty on cobras was introduced to reduce the population — and instead incentivized people to breed them.

In detection engineering, the equivalent is rewarding rule count as a proxy for coverage. Engineers respond rationally to the incentive and ship more rules — but those rules are low-fidelity, poorly tuned, and increase analyst fatigue without meaningfully improving detection capability. The coverage metric looks healthy; the program's actual ability to detect threats has degraded.

Any metric tied to volume — rules deployed, alerts generated, playbooks written — carries Cobra Effect risk if it is not balanced against a quality signal.

What This Means for Your Program

The sections that follow provide a catalog of metrics worth tracking across the Primary, Secondary, and Tertiary tiers. Not all of them will be right for your program at every stage of maturity, and not all of them need to be reported to every audience. Use them as a starting point, not a checklist.

The visibility matrix accompanying each metric is designed to help you ask the right question before you report anything: who needs this number, what decision does it support, and what behavior does tracking it incentivize? If you cannot answer all three, the metric is not ready to be a target.

How to Read This Catalog

Each metric in this catalog is presented in a consistent format — a short narrative description followed by a detail table. The table fields are standardized across every metric so the document can be used as a quick reference as well as a guide. Here is what each field means:

Definition — what the metric measures in plain language.

Formula — the mathematical expression used to calculate the metric. Some metrics include multiple formulas where different calculation methods apply.

Tier — whether the metric belongs to the Primary, Secondary, or Tertiary tier as defined in the Metric Tiers section.

Type — whether the metric is a KPI, KRI, or Operational metric as defined in the KPIs vs KRIs vs Operational Metrics section.

Preferred Collection Method — how the metric should ideally be collected in a healthy program:

• Ad-Hoc describes any metric that requires manual intervention to collect, even if parts of the process are automated. If an analyst must touch, pull, transform, or assemble data at any point in the process, the metric is ad-hoc. This is not a permanent state — it is a signal that instrumentation or automation is missing.
• Automatic describes any metric that can be produced and delivered without manual intervention. The underlying data is instrumented, processed, and reported automatically. Automatic is the target state for all metrics in a mature detection program.

Cadence — how frequently the metric should be reported or reviewed.

Visibility — which audiences should receive or have access to this metric.

Dependencies — the data sources, integrations, platform capabilities, or program conditions that must be in place before the metric can be collected. A metric that cannot be collected because its foundational data does not exist is not a gap in reporting — it is a gap in instrumentation. Dependencies are broken into layers where relevant:

• Analytical Layer — the platforms or systems where the data lives
• Key Fields — the specific data fields required to calculate the metric
• Requirements — any additional conditions or instrumentation that must be in place

Notes — any critical context, caveats, cross-references to related metrics, or pitfalls specific to that metric.

Primary Metrics

Primary metrics are the core indicators of a threat detection program's effectiveness. They directly measure the program's ability to identify, alert on, and respond to real threats in a timely and accurate manner. These metrics align closely with organizational risk reduction goals and are reported to leadership and stakeholders as key performance indicators (KPIs).

True Positive Rate (TPR)

The True Positive Rate measures the percentage of tickets confirmed as actual security threats. It is the clearest signal of detection fidelity — how well the program distinguishes real threats from noise.

False Positive Rate (FPR)

The False Positive Rate measures the percentage of alerts or tickets that were triggered but later identified as benign or not security-relevant. A high FPR is a direct driver of alert fatigue, eroding analyst trust in the detection program and reducing overall SOC efficiency over time.

Mean Time to Detect (MTTD)

Mean Time to Detect measures the average time from when a threat event first occurs in the environment to when an analyst or automated agent is assigned and actively reviewing it. This metric captures the full detection chain — from raw event to human eyes — and is one of the most direct indicators of how quickly the program surfaces real threats.

Mean Time to Respond (MTTR)

Mean Time to Respond measures the average time from ticket disposition to response completion. While MTTR is primarily a CSIRT metric, it is important for the detection engineering team to track — it provides visibility into how detections perform end-to-end and whether the program is contributing to timely threat resolution.

Automated Response Rate (ARR)

Automated Response Rate measures the percentage of tickets where automation fully or partially determined the outcome through disposition analysis. This metric is a direct indicator of automation maturity — tracking both rates separately reveals how much of the ticket lifecycle the program has truly automated versus how much still relies on human decision making.

False Positive Reduction (FPR Reduction)

False Positive Reduction measures the percentage decrease in false positives over a defined period as a result of automated enrichment and triage improvements. Unlike FPR which is a point-in-time snapshot, this metric is a continuous improvement trend — tracked week over week or month over month to validate that automation and tuning investments are meaningfully reducing noise over time.

Secondary Metrics

Secondary metrics are supporting indicators that provide context and insight into the factors influencing primary metric outcomes. While they do not directly measure detection effectiveness, they offer visibility into breadth, quality, coverage, and performance across systems and processes. These metrics help diagnose gaps, justify tuning efforts, and inform decision-making around detection strategy.

Detection Coverage

Detection Coverage measures the proportion of relevant MITRE ATT&CK techniques that are monitored by active detection rules in the environment. It is broken down into three sub-metrics — Tactic Coverage, Technique Coverage, and APT Coverage — each providing a different lens on how broadly and specifically the detection program addresses the threat landscape. Together they answer the question: what general coverage do we have over adversaries?

Important: Coverage does not translate to detection success. The adversary landscape is continuously evolving — new tactics, techniques, and procedures emerge regularly, making complete coverage an unrealistic and misleading target. Coverage metrics should instead be treated as operational intelligence on your detection library: a tool for understanding potential gaps, framing detection objectives, and informing engagement activities such as Purple Teaming exercises. They also provide a level of traceability between detections and the adversarial techniques they are designed to address. Use coverage to drive conversations, not to declare victory.

MITRE ATT&CK Coverage

MITRE ATT&CK Coverage measures the percentage of MITRE ATT&CK techniques within each tactic that have active detection logic mapped to them. Coverage at this level provides a broad view of where the detection program has visibility and where entire adversary objectives may go unmonitored.

Detection Fidelity

Detection Fidelity is a composite metric that reflects the reliability, confidence, and actionability of a detection rule. Rather than measuring a single data point, it combines multiple signals — data source quality, historical trigger rate, and true and false positive rates — into a single score per rule. This gives detection engineers a normalized view of how trustworthy any given detection is and where tuning investment is most needed.

Detection Latency

Detection Latency measures the time delay between when an event occurs in the environment and when it is processed and alerted by the detection system. This is a purely infrastructure and pipeline metric — it isolates the detection system's processing speed from any human or operational factors. Unlike MTTD which measures the full delivery chain to an analyst, Detection Latency ends the moment the alert fires.

Incident Conversion

Incident Conversion measures the total number of tickets that are confirmed as incidents over a given period. A ticket converts to an incident when it is validated as a real threat requiring formal response — typically reflected through a priority escalation or disposition change in the case management system, such as a ticket moving from a sub-priority alert to a Sev1. This metric provides visibility into how many detections are resulting in real incidents and is a direct signal of threat activity in the environment.

Time to Tune

Time to Tune measures the average duration from when a tuning recommendation is made to when it is implemented. It is a direct measure of how quickly the detection engineering team responds to and resolves detection debt. As technology and automation capability evolve, the workflow that drives this metric changes — but the formula remains consistent regardless of how the recommendation was generated or how the implementation was executed.

Control Mapping Rate

Control Mapping Rate measures the percentage of critical security controls that are mapped to at least one active detection rule, ensuring the detection program maintains alignment with the organization's control framework. For organizations without a formally adopted control framework, detection rules can be aligned to security domains such as Network, Endpoint, Web, SaaS, or Identity as a proxy — achieving the same goal of traceability between detections and what they are designed to protect.

Log Source Coverage

Log Source Coverage measures the percentage of ingested log sources that are actively referenced in detection logic. Ingestion alone does not constitute coverage — a log source sitting in the SIEM or data platform without a single detection rule referencing it provides no operational value to the detection program. This metric surfaces that gap.

Playbook Success Rate

Playbook Success Rate measures the percentage of playbook executions that complete without a technical failure. A failure is defined as any execution that errors out due to an integration failure, API timeout, or platform issue — not a manual override or analyst intervention. This is a technical health metric for the automation layer of the detection program.

Average Playbook Execution Time

Average Playbook Execution Time measures the average duration of automated workflow executions. Tracked at two levels — program-wide and per playbook — the program-wide average reflects the overall health and performance of the automation layer, while per-playbook tracking identifies long-running workflows that may be introducing delays in detection response or masking performance issues in the aggregate.

Integration Health Score

Integration Health Score measures the availability of each integration that SOAR workflows and detection delivery pipelines depend on, tracked as an uptime percentage per integration. It surfaces integration failures that may be silently impacting playbook execution, detection alerting, and automation reliability across the full detection delivery chain.

Manual Touchpoints

Manual Touchpoints measures the average number of human interventions per playbook workflow that is not fully automated to resolution. Fully automated workflows that close without any human involvement are excluded — this metric is scoped specifically to workflows where automation falls short of full resolution and a human must intervene to complete the process. It is the inverse signal of Automated Response Rate: where ARR measures how much automation is resolving, Manual Touchpoints measures how much human effort remains in the workflows that are not yet fully automated.

Tertiary Metrics

Tertiary metrics provide operational insights that inform on the maturity, hygiene, and developmental aspects of the threat detection program. These metrics are used internally by detection engineering teams to optimize processes, track backlog and rule lifecycle, and measure long-term improvements. They are important for maintaining a healthy detection program but are not typically used as primary indicators of success.

Volume Metrics

Volume Metrics capture the daily counts of all inputs and outputs across the detection pipeline. Each metric is tracked as both a raw daily count and an average daily rate over a longer period — the count surfaces anomalies and spikes on a given day, while the trend rate reveals whether volume is growing, stable, or shrinking over time. Together they provide operational awareness of system load, analyst workload, and pipeline activity.

Shared Formula Pattern

Shared Table Fields

Event Volume

The total number of events across all data points monitored by detections. This is the broadest input metric in the pipeline — it reflects the raw volume of data the detection program is processing and provides the baseline against which all downstream volume metrics are contextualized.

Detection Volume

The total number of detections that execute daily. This metric reflects the operational load on the detection engine and provides context for downstream alert and ticket volumes.

Alert Volume

The total number of alerts generated across detection rules daily. Alert Volume is the first downstream output of detection execution and is a key signal for analyst workload and detection noise.

Ticket Volume

The total number of tickets generated across alerts daily. Ticket Volume reflects the downstream output of alert triage and is a direct gauge of analyst workload and system noise reaching the case management layer.

Ignore Event Volume

The total number of detections that are prevented from creating tickets daily. This metric captures the volume of events that the detection program intentionally suppresses before they reach the ticketing layer — providing visibility into how much noise is being filtered out of the analyst workflow.

Automation Usage

The total number of automations executed daily. This metric tracks the operational load on the automation layer and provides a baseline for understanding how actively SOAR workflows are being utilized across the detection pipeline.

Detection Conversion

Detection Conversion measures how volume moves through each stage of the detection pipeline — from detection execution to alert generation to ticket creation. Tracked as three separate conversion rates, it reveals where drop-off is occurring in the pipeline and whether detections are successfully surfacing through to analyst hands.

Stale Count

Stale Count measures the number of detections or automations that have not been reviewed, tested, or updated within a defined time window — typically six months. Stale detections and automations are a hygiene and risk signal: they may no longer reflect the current threat landscape, environment, or data sources, leading to blind spots or unreliable automation behavior.

Time to Triage

Time to Triage measures the average time taken by an analyst to disposition an alert from the moment it is assigned. It is a direct reflection of the quality and completeness of the data provided with each alert — well-enriched alerts with clear context are triaged faster than alerts that require manual investigation to understand.

Development Time

Development Time measures the average time it takes to build, test, and deploy a new detection or automation from concept to production. It is a measure of engineering throughput and process efficiency — a long development time may indicate bottlenecks in the build, test, or review pipeline.

Suppression Ratio

Suppression Ratio measures the total count of suppressions applied per detection, broken down by Manual and Automated methods. It provides visibility into which detections are generating the most suppression activity and whether that suppression is being applied by automation or by analysts.

Detection Effectiveness

Detection Effectiveness measures the percentage of detection rules that successfully trigger during red team simulations, atomic testing, or validation exercises. It is the only metric in this catalog that validates detection capability through active testing rather than production data — providing ground truth on whether detections work as designed.

Coverage Drift

Coverage Drift measures the change in detection coverage over time — indicating whether detection capability is expanding or regressing between periods. A positive value means coverage is growing; a negative value means coverage has decreased and warrants investigation.

Deployment Frequency

Deployment Frequency measures how often new detections, automations, playbooks, and integrations are successfully deployed to production within a given timeframe. It is a key indicator of engineering agility and the team's ability to deliver capability improvements continuously. Tracked across four sub-categories, it provides a complete picture of output across all program components.

Detections Created

Automations Created

Playbooks Created

Integrations Created

Change Frequency

Change Frequency measures how often existing detections, automations, or playbooks are modified in a given period. It indicates responsiveness to threat evolution and process improvement — a healthy change frequency reflects an active, adaptive detection program rather than a static one.

Task Automation Ratio

Task Automation Ratio measures the percentage of tasks automated versus total tasks per incident type, specifically for ad-hoc automation commands and scripts executed manually by analysts within a case or incident — outside of structured playbook workflows. This metric lives in the bounds of the case management system and reflects analyst-initiated automation rather than SOAR-orchestrated automation.

Platform Uptime

Platform Uptime measures the operational reliability of the detection and SOAR platforms, tracked as an availability percentage per platform. It is the foundational health metric for the entire detection program — if the platforms are unavailable, no other metric in this catalog can be trusted.

Investigation Duration

Investigation Duration measures the average time elapsed from analyst assignment to case resolution, scoped per detection rule and per case. It is a proxy for detection complexity — a rule that consistently produces long investigations is a signal that the alert may lack sufficient context, the rule logic may be too broad, or there is insufficient automation supporting the analyst through the workflow. Coupling this metric with rules that have no playbook coverage surfaces the highest-friction detections in the program and points directly at where engineering investment would have the greatest return.

• Detection Engineering & Threat Hunting SIG
  • Detection Engineering Within the Enterprise Security Ecosystem
  • Detection Engineering Metrics Catalog
  • Detection Outcomes and Tiering